Enhanced volume viewing

ABSTRACT

A method for improved 3D imaging of a volume is disclosed. The application discussed is improved assessment of complex 3D structures including breast microcalcifications and incorporation of geo-registered tools. The 3D imaging is displayed on a geo-registered head display unit.

CROSS-REFERENCES TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 16/594,139 filed on Oct. 7, 2019, which claims the benefit ofU.S. Provisional Application 62/743,837 filed on Oct. 10, 2018. Thepresent application is also a continuation-in-part of U.S. patentapplication Ser. No. 15/949,202 filed on Apr. 10, 2018.

TECHNICAL FIELD

Aspects of this disclosure are generally related to radiologicalimaging, and more particularly to 3D breast imaging.

BACKGROUND

An improved type of mammogram to detect breast cancer is referred to as3D mammogram. A better description is digital breast tomosynthesis(DBT). DBT differs from a common mammogram in that with DBT, the X-raymachine sweeps out an arc taking multiple X-rays while the breast iscompressed by a paddle against the X-ray detector plate whereas astandard mammogram takes two X-ray of the compressed breast from thevertical and oblique angles. Thus, the DBT process takes X-ray picturesfrom more angles and in this process, there are fewer false positives(of cancer) and fewer call backs for additional evaluation are reducedsaving costs to the medical system and angst of the part of the patient.

There is, however, a problem with both the DBT and the common mammogram.Specifically, neither process can reliably discern the patterns ofmicrocalcifications within the breast—some of which are indicative ofductal carcinoma in situ (DCIS) and some of which are benign. Ingeneral, these microcalcifications may appear as a cluster, but it canbe difficult to determine the exact 3D distribution of the cluster ofmicrocalcifications. When one considers that the number of thesemicrocalcifications typically ranges between as small as single digit upto 1000 or more, seeing all of these calcifications on a single or,multiple in the case of DBT, two dimensional arrays can present a majorchallenge to the radiologist to discern a branching structure which issuspicious for DCIS.

Known techniques for 3D viewing of medical images are described in U.S.Pat. No. 9,349,183, Method and Apparatus for Three Dimensional Viewingof Images, issued to Douglas, U.S. Pat. No. 8,384,771, Method andApparatus for Three Dimensional Viewing of Images, issued to Douglas,Douglas, D. B., Petricoin, E. F., Liotta L., Wilson, E. D3D augmentedreality imaging system: proof of concept in mammography. Med Devices(Auckl), 2016; 9:277-83, Douglas, D. B., Boone, J. M., Petricoin, E.,Liotta, L., Wilson, E. Augmented Reality Imaging System: 3D Viewing of aBreast Cancer. J Nat Sci. 2016; 2(9), and Douglas, D. B., Wilke, C. A.,Gibson, J. D., Boone, J. M., Wintermark, M. Augmented Reality: Advancesin Diagnostic Imaging. Multimodal Technologies and Interaction, 2017;1(4):29.

SUMMARY

All examples, aspects and features mentioned in this document can becombined in any technically possible way. All patents and patentapplications referenced herein are incorporated by reference.

Aspects of the present disclosure include an improved process forevaluating the DBT data than looking at 2D image slices with a cloud ofmicrocalcifications and/or tumorous tissue. In some implementations thatimproved process includes looking at a geo-registered 3D DBT volume bytrue stereoscopic viewing and the ability to rotate, zoom, and fly intothis volume. This enables the medical person viewing the 3D DBT data togain a thorough understanding of the underlying structure of themicrocalcifications and/or tumorous tissue. Additionally, the breast iscompressed in multiple different manners and the morphologic changesanalyzed. Apparatus and methods by which this better way can be achievedare described herein.

Some implementations comprise geo-registration and display of digitaltomosynthesis (DBT) data in three dimensions (3D). From the 2D DBT dataa 3D volume will be created and presented to medical personnel in truestereoscopic images via an augmented reality/virtual reality (AR/VR)headset. Note that the images obtained during the digital tomosynthesisprocess are images with differing gray scales depending on the densityof the tissue attenuated by the X-ray beam onto the detector array foreach respective pixel of the array.

Some implementations comprise but, are not limited to, an apparatuscomprised of a computer system which runs the mathematical process(described below) applied to the DBT data and generates stereoscopicimages of 3D DBT data, a medical personnel control device (e.g., but notlimited to a joy stick or game controller developed in accordance withU.S. Ser. No. 16/524,275 USING GEO-REGISTERED TOOLS TO MANIPULATETHREE-DIMENSIONAL MEDICAL IMAGES) from which commands are issued tochange viewing points, rotate images, etc. (described below) andaugmented reality (AR)/virtual reality (VR) headset which displays the3D DBT data in true stereoscopic format. Further, some implementationswill have a geo-registered focal point pen and a geo-registeredpedestal.

Some implementations comprise, but are not limited to, a set of pin headsized radiodense surfaces (nominally eight) placed on the skin surfaceof each breast in a manner such that the majority of the tissue of thebreast would be encased within the volume subtended by radiodensepoints. These pin head radiodense surfaces are affixed to the breastprior to commencing the DBT process and would have a unique signaturewhen projected onto the DBT system detector array. This would beaccomplished in accordance with procedures outlined in U.S. Ser. No.16/509,592 IMPLANTABLE MARKERS TO AID SURGICAL OPERATIONS.

Some implementations comprise but, are not limited to, establishing a 3Dgrid system and associated X, Y, Z coordinate system with a resolutionconsistent with the digital tomosynthesis system on which the DBT datais collected. In accordance with this process, the tissue within theimages generated by the digital tomosynthesis would be geo-registeredwithin the grid system and associated X, Y, Z coordinate system. Thisgrid would include multiple radiographically detectable markers. Theindividual markers could be made of the same or different size, shape,materials as desired to assist with geo-registration. A unique signatureof a particular spot may be use to track a portion of the breast under afirst and second configuration. Thus, the internal anatomic features canbe registered to the external grid features.

Some implementations comprise, but are not limited to, establishing amathematical process by which each of the microcalcifications and/ortumorous tissue could be geo-registered within the 3D grid system andassociated X, Y, Z coordinate system. In this mathematical process, notethat the X-ray for each of the X-ray pictures is approximated to be apoint source. When the X-ray passes through the microcalcificationand/or tumorous tissue, the X-ray beam is partially attenuated and isprojected onto the detector array (i.e., the microcalcification and/ortumorous tissue will have a specific X-coordinate and a specificY-coordinate on the detector array). These microcalcifications and/ortumorous tissue specific X-coordinates and a specific Y-coordinates willchange as the X-ray machine progresses through the arc and subsequentX-ray pictures are taken. First consider the X-ray machine in thevertical position over the breast and a single microcalcification in thecenter of the breast. The projection of this microcalcification onto thedetector array would be in the center of the detector array. If themicrocalcification were located off of the center line, the projectionof this microcalcification on the detector array would be based on theangle from the X-ray tube through the X-coordinate, Y-coordinate andZ-coordinate of the microcalcification and onto the detector array. Asthe X-ray machine moves to a new position and takes the next X-ray, themicrocalcification will have a new X-coordinate and a new Y-coordinateon the detector array. By back plotting a line at the angle from thefirst X-coordinate and Y-coordinate to the point of emission of theX-ray and next back plotting the line at the angle from the secondX-coordinate and second Y-coordinate to the point of emission of theX-ray, these two lines will intersect at the X-coordinate, Y-coordinateand Z-coordinate of the microcalcification. This process would berepeated multiple times for each of the X-rays within the DBTexamination and with these further calculations, the estimate ofX-coordinate, Y-coordinate and Z-coordinate of the microcalcificationscan be refined. Given that the X-coordinate, Y-coordinate andZ-coordinate of the microcalcification are calculated and object toimage distance is known, this process would also be able to correct formagnification, focal spot blurring and motion blurring.

Some implementations comprise, but are not limited to, establishing(i.e., plotting digitally) the locations of the microcalcifications,normal breast tissue, blood vessels, and/or tumorous tissue discoveredin the above mathematical process in the grid system with theirrespective X, Y, Z coordinates.

In some implementations, the medical personnel could select viewingoptions via the control unit.

Some implementations comprise, but are not limited to, a process bywhich the 3D DBT data can be viewed. Specifically, a view point, whichcan be modified during the course of the examination, will beestablished. From this view point two different images will begenerated—one from the position of the left eye viewing point (LEVP) anda separate one for the right eye viewing point (REVP). These images willbe projected on AR/VR headset. The 3D volume could: be rotated in anydirection; zoomed in/out; false color could be added for DBT data withintensity indicative of microcalcification and/or tumorous tissue typetissue; and the viewing point could be modified and new images generatedfor the LEVP and REVP would be generated by these changes accordingly(U.S. Pat. No. 8,384,771, Method and apparatus for three dimensionalviewing of images and U.S. Pat. No. 9,349,183, Method and apparatus forthree dimensional viewing of images).

In some implementations, a 3D cursor would be generated in accordancewith U.S. Pat. No. 9,980,691, Method and apparatus for three dimensionalviewing of images and U.S. patent application Ser. No. 15/878,463,Interactive 3D cursor for use in medical imaging to assist medicalpersonnel in viewing a sub-set of the DBT data. The 3D cursor could bechanged in size and shape and moved within the 3D DBT volume to alocation and orientation selected by the medical personnel via commandsinserted to the computer (or interface to cloud computing) via thecontrol unit. At the discretion of the medical person viewing the 3D DBTdata, tissue external to the 3D cursor could be temporarily removed toimprove viewing.

In some implementations, filtering of DBT data could be applied to theentire 3D volume or to a sub-set of data contained within the 3D cursor(U.S. patent application Ser. No. 15/904,092, INTERACTIVE VOXELMANIPULATION STRATEGIES IN VOLUMETRIC MEDICAL IMAGING ENABLES VIRTUALMOTION, DEFORMABLE TISSUE, AND VIRTUAL RADIOLOGICAL DISSECTION). Thisfiltering could largely eliminate non-microcalcification and/or tumoroustype tissue and reduce/eliminate occlusion of microcalcifications and/ortumorous type tissue for enhanced viewing by medical personnel.

In some implementations, a focal point pen would be generated and movedwithin the 3D DBT volume. The focal point pen could be used inter aliato: mark/trace the microcalcification and/or tumorous tissue structure.The focal point pen could create virtual symbols (e.g., arrows tostructures of interest or virtual written notes of findings) to:facilitate discussions between medical personnel and/or medicalpersonnel with patients; and assist in preparation of reports. The focalpoint pen would be generated in accordance with U.S. patent applicationSer. No. 16/524,275 USING GEO-REGISTERED TOOLS TO MANIPULATETHREE-DIMENSIONAL MEDICAL IMAGES.

In some implementations, a geo-registered pedestal would be availablefor the medical personnel. In this implementation, the medical personcould move the 3D cursor to a region of interest, re-size/reshape the 3Dcursor to capture tissue of interest, and then move the contents to thegeo-registered hand-held pedestal and affix the contents of the 3Dcursor to the pedestal. This would enable the medical person closelyexamines the tissue as he/she moves their hand holding the pedestal andits contents. The geo-registered hand-held pedestal would be generatedin accordance with U.S. patent application Ser. No. 16/524,275 USINGGEO-REGISTERED TOOLS TO MANIPULATE THREE-DIMENSIONAL MEDICAL IMAGES.

In some implementations, given that the X-coordinate, Y-coordinate andZ-coordinate of the microcalcification are calculated and object toimage distance is known, this process would also be able to correct formagnification, focal spot blurring and motion blurring.

Another implementation is performing compression at varying levels andgenerating a 3D dataset at each compression level. This would allow theradiologist to better understand how the tissues in the body change inrelation to the changing pressures. This may help in determining whethera tumor is benign or malignant.

Some implementations comprise: a computer (or interface to cloudcomputing) which applies software to digital tomosynthesis medicalimages at the direction of medical personnel reviewing 3D medicalimages; a head display unit (or other type display which provides truestereoscopic imagery) that presents the geo-registered 3D DBT medicalimage(s); a geo-registered focal point pen that interacts withgeo-registered 3D DBT medical images; a geo-registered pedestal/platformthat interacts with geo-registered 3D DBT medical images; ageo-registration master control platform component interacts with othercomponents and, in conjunction with geo-processing software would enablegeo-registration of these components within the overall systemcontaining the volumetric DBT medical images, and other systemcomponents; and, software for geo-registration of components within thesystem and generation of images to be displayed on the head displayunit.

Some implementations comprise, but are not limited to, establishing(i.e., plotting digitally) the locations of the microcalcificationsand/or tumorous tissue discovered in the above mathematical process inthe grid system. This 3D DBT data would then be available for viewingvia the head display unit.

Some implementations comprise, but are not limited to, master controlplatform functionality for medical personnel reviewing 3D DBT medicalimages to direct digital displays to be presented on the head displayunit. This functionality includes that previously disclosed in U.S. Pat.No. 8,384,771, Method and apparatus for three dimensional viewing ofimages, U.S. Pat. No. 9,349,183, Method and apparatus for threedimensional viewing of images, U.S. Pat. No. 9,473,7166 Method andapparatus for three dimensional viewing of images, and U.S. Ser. No.16/524,275 USING GEO-REGISTERED TOOLS TO MANIPULATE THREE-DIMENSIONALMEDICAL IMAGES. These additional functions and associated commands mayinclude, but are not limited to, the following functions to facilitatethe examination of geo-registered 3D DBT medical images by medicalpersonnel reviewing geo-registered 3D DBT medical images: wheninterfacing with the geo-registered 3D cursor, change the location andorientation of the cursor; invoke convergence; invoke filtering,segmentation, sequencing, statistical, and reporting operations; invokegeo-registration system consisting of: head mounted display withseparate eyepiece displays for true stereoscopic displays; ageo-registered focal point pen; a geo-registered pedestal/platform;geo-registration to interact with the computer, in conjunction withgeo-processing software which would enable geo-registration of thesecomponents within the overall system containing the volumetric 3D DBTmedical images, and other system components; invoke movement optionsfor: the head mounted display with separate eyepiece displays for truestereoscopic displays; a geo-registered focal point pen; ageo-registered pedestal/platform; in conjunction with geo-processingsoftware which would enable geo-registration of these components withinthe overall system containing the volumetric 3D DBT medical images, andother system components; invoke a 3D cursor which would begeo-registered and moved within the coordinate system to volumes ofinterest/concern copy the volume and transport it to the pedestal fordetailed examination; input the volume subtended by the 3D cursor intomachine learning and artificial intelligence algorithms in accordancewith PCT/US19/23968 RADIOLOGIST-ASSISTED MACHINE LEARNING WITHINTERACTIVE, VOLUME-SUBTENDING 3D CURSOR.

Some implementations may include a 3D cursor generated in accordancewith U.S. Pat. No. 9,980,691 and U.S. patent application Ser. No.15/878,463 to assist medical personnel in viewing a sub-set of the DBTdata. The 3D cursor could be changed in size and shape and move withinthe 3D DBT volume to a location selected medical personnel via commandsinserted to the computer via the control unit. At the discretion of themedical person viewing the 3D DBT data, tissue external to the 3D cursorcould be temporarily removed.

Some implementations may comprise, but are not limited to, filtering of3D DBT data: applied to the entire 3D volume or; to a sub-set of datacontained within the 3D cursor (See U.S. patent application Ser. No.15/904,092, Processing 3D medical images to enhance visualization). Thisfiltering could largely eliminate non-microcalcification type tissue andeliminate occlusion of microcalcifications for enhanced viewing bymedical personnel.

Some implementations comprise a focal point pen. The focal point pencould be used inter alia to: mark/trace the microcalcification and/ortumorous tissue structure. The focal point pen could create virtualsymbols (e.g., arrows to structures of interest or virtual written notesof findings) to: facilitate discussions between medical personnel and/ormedical personnel with patients; and assist in preparation of reports.The focal point pen would be generated in accordance with U.S. Ser. No.16/524,275, USING GEO-REGISTERED TOOLS TO MANIPULATE THREE-DIMENSIONALMEDICAL IMAGES.

Some implementations comprise a hand held geo-registered pedestal wouldbe generated to mount the contents of the 3D cursor to facilitateexamination 3D DBT results of the breast by medical personnel inaccordance with U.S. patent application Ser. No. 16/524,275, USINGGEO-REGISTERED TOOLS TO MANIPULATE THREE-DIMENSIONAL MEDICAL IMAGES.

Some implementations comprise, but are not limited to, implementation offunctionality in viewing the 3D DBT data. The medical personnel couldselect viewing options via the control unit. These options would includebut, are not limited to: selection of viewing points; rotation of the 3DDBT data; zooming into the 3D DBT data; addition of false color tocertain types of tissue bases on its intensity; flying into the 3D DBTvolume. The entire volume can then be viewed stereoscopically or somesubset of the data within the 3D cursor. The 3D cursor could be changedin size and shape and move within the 3D volume to a location ororientation selected medical personnel via commands inserted to thecomputer via the control unit. Filtering could be applied to reduceocclusion by non-microcalcification type tissue. The focal point pencould be used inter alia to: mark/trace the microcalcificationstructure; assist with eye tracking and focal point focusing; insertnotes for to future reference to facilitate discussions between medicalpersonnel and patients; and assist in report preparation. The contentsof the 3D cursor could be moved and affixed to the hand heldgeo-registered pedestal.

Some implementations comprise, but are not limited to, software thatimplements the following: geo-registration of DBT data throughestablishing a 3D grid system and associated X, Y, Z coordinate system;the mathematical process to convert the 2D DBT data into a 3D volume;the plotting the 3D DBT data; the master control platform functionality;the process by which the 3D DBT data can be viewed; establishing the 3Dcursor with associated functionality; the process by which the 3D DBTdata can be filtered; establishing the geo-registered focal point withassociated functionality; establishing the geo-registered pedestal withassociated functionality; implementing the overall system functionality.

Some implementations comprise visualization and analysis of anatomicstructures other than in the breast. For example, the same type analysiscould be performed for soft tissue lesions elsewhere in the body.Alternatively, this same analysis process could be performed for other3D imaging datasets for analysis, to include imaging of inanimateobjects.

Some implementations comprise but, are not limited to applying differentlevels of pressure to compress the breast. This digital tomosynthesismethod could include applying the standard pounds per square inch (psi)for the initial reading (inter alia for comparative purposes withprevious tomosynthesis readings) and subsequently increasing thepressure by a step function increase for a secondary reading. A third(or more) step increases could also be applied. In a similar manner, thepsi could be increased is a near continuous fashion with DTS applicationreadings taken during the periods of time when the psi was beingincreases. Technique would be optimized, of course, to minimize motionartifact. Note that different types of breast tissue respond differentlywith differing levels of pressure: dense breast tissue does not compressas easily as breast tissue with a high fatty tissue content. Further,different tumor types compress and are distorted in a different mannerunder changes in external pressure. These different levels can assistwith more accurate diagnoses.

Some implementations comprise, but are but limited to obtainingquantification/measurement of image data during the application ofdiffering levels of psi. The methods of obtainingquantification/measurement image data include, but are not limited tothe following: calculating the number of voxels contained in the volumesubtended by the radio dense markers and length, width, and height databetween radio dense markers; use of a 3D cursor(s) would be generated inaccordance with U.S. Pat. No. 9,980,691, Method and apparatus for threedimensional viewing of images and U.S. application Ser. No. 15/878,463,Interactive 3D cursor for use in medical imaging to assist medicalpersonnel in viewing the DTS data to encase the tumor(s) and calculatingthe number of voxels contained in the 3D cursor(s); volume (note thatthe segmentation and filtering process of U.S. Pat. No. 8,384,771,Method and apparatus for three dimensional viewing of images could beused to eliminate non-tumor tissue contained within the 3D cursor(s);and measurements of distortion of the tumor size and shape.

BRIEF DESCRIPTION OF FIGURES

FIG. 1 is a flow diagram that illustrates a process of generating 3Dvolumetric dataset from a digital breast tomosynthesis dataset and usingthis dataset for enhanced viewing.

FIG. 2 illustrates an apparatus for implementing the process of FIG. 1.

FIG. 3A illustrates the breast with georegistration points.

FIG. 3B illustrates the breast with georegistration points and ageoregistration grid.

FIG. 3C illustrates the breast with georegistration points,georegistration grid and the coordinate system.

FIG. 4A illustrates an initial configuration of the X-ray detector,x-ray beam, breast, microcalcification within the breast, the detectorand a first image.

FIG. 4B illustrates movement of the X-ray detector and subsequent image.

FIG. 5A illustrates the geometry of the X-ray tube, X-ray detector,compression paddle, breast and microcalcification.

FIG. 5B illustrates the geometry of the microcalcification within thebreast, the photon beams from the two X-ray tube positions, the X-raydetectors.

FIG. 6A illustrates an artery, a vein, breast tissue,microcalcifications, and a 3D cursor.

FIG. 6B illustrates filtering (i.e., subtraction) of the tissuesexternal to the 3D cursor.

FIG. 6C illustrates changing the transparency of the tissues within the3D cursor.

FIG. 6D illustrates filtering (i.e., subtraction) of all tissues bothinside and external to the 3D cursor with the exception of the breastmicrocalcifications.

FIG. 7A illustrates the initial viewing perspective.

FIG. 7B illustrates changing the interocular distance and angular fieldof view.

FIG. 7C illustrates the volume of interest (VOI) contained inside of the3D cursor rotated 90 degrees.

FIG. 7D illustrates changing the viewing perspectives by rotating it by90 degrees.

FIG. 8A illustrates the 3D cursor affixed to a geo-registered platform.

FIG. 8B illustrates tilting of the geo-registered platform.

FIG. 8C illustrates pointing to the microcalcifications inside of the 3Dcursor with the geo-registered focal point pen.

FIG. 9 illustrates a top down view of the mammographer's deskillustrating several of the tools with position and orientationtracking.

FIG. 10 illustrates the conversion of a digital breast tomosynthesisdataset into a single voxelated dataset.

FIG. 11A illustrates a low pressure of compression of the breast.

FIG. 11B illustrates a high level of compression of the breast.

FIG. 11C illustrates an initial configuration of a breast mass.

FIG. 11D illustrates a subsequent configuration of a breast mass.

FIG. 12A illustrates a low level of compression of the breast and abreast mass that is round in shape.

FIG. 12B illustrates a high level of compression of the breast and thebreast mass remains round in shape.

FIG. 12C illustrates a low level of compression of the breast and abreast mass that is round in shape.

FIG. 12D illustrates a high level of compression of the breast and thebreast mass becomes flattened.

FIG. 13 illustrates a digital breast tomosynthesis dataset performedwith skin markers.

DETAILED DESCRIPTION OF FIGURES

FIG. 1 is a flow diagram that illustrates a process of generating 3Dvolumetric dataset from a tomosynthesis dataset and using this datasetfor enhanced viewing. The flow diagrams do not depict the syntax of anyparticular programming language. Rather, the flow diagrams illustratethe functional information one of ordinary skill in the art requires tofabricate circuits or to generate computer software to perform theprocessing required in accordance with the present invention. It shouldbe noted that many routine program elements, Such as initialization ofloops and variables and the use of temporary variables are not shown. Itwill be appreciated by those of ordinary skill in the art that unlessotherwise indicated herein, the particular sequence of steps describedis illustrative only and can be varied without departing from the spiritof the invention. Thus, unless otherwise stated the steps describedbelow are unordered meaning that, when possible, the steps can beperformed in any convenient or desirable order. The first step 100 is torecord precise geometry of the digital breast tomosynthesis equipment.The second step 101 is prior to commencing digital tomosynthesis exam,have the option to affix pin head size radiographically detectablemarkers on the surface of the patient's breast. The preferred optionwould be to have the mammography technologist place the markers, seeU.S. patent application Ser. No. 16/509,592, IMPLANTABLE MARKERS TO AIDSURGICAL OPERATIONS. The third step 102 is to perform digital breasttomosynthesis examination and collect data. Note that this can beperformed under varying levels of compression of the breast. Forexample, a first 3D volume can be performed at a first level ofcompression of the breast. Then, a second 3D volume can be performed ata second level of compression of the breast. The fourth step 103 is todownload DBT data into the 3D processing system along with associatedmeta data for particular DBT System including DBT system resolution, arcdegrees, number of images taken. The fifth step 104 is to create a gridand associated X, Y, Z coordinate system which is consistent with theDBT system resolution and subtends the volume subtended by the pin headsize radiographically detectable markers. The sixth step 105 is to runthe mathematical process to convert the multiple 2D DBT images into asingle 3D DBT dataset composed of voxels with each voxel having a unique(x, y, z) coordinate. The seventh step 106 is to plot the 3D DBT data inthe X, Y, Z coordinate system. The eighth step 107 is to display the 3DDBT data in true 3D via an extended reality (e.g., augmented reality,mixed reality or virtual reality) headset for radiologist examination.The ninth step 108 is for the computer responds to radiologist commandsissued via the control unit to invoke the following: establishing viewpoint; rotating, zooming, flying through the 3D volume and/or addingfalse color to denote selected tissue types; invoking tissue filteringto improve visualization of microcalcifiction and/or tumerous tissue;creation of a 3D cursor and movement thereof to regions of interest andre-size/re-shape, as desired; remove tissue external to 3D cursor, asdesired; positioning the focal point pen to tissue of interest andcreate symbols/notes, as desired; move contents of 3D cursor togeo-registered pedestal and affix contents to hand held pedestal; and,move hand held pedestal with affixed contents, as desired. View thereconstructed 3D dataset via standard slice-by-slice scrolling or viaadvanced imaging techniques including those described in U.S. Pat. No.8,384,771, Method and apparatus for three dimensional viewing of images,U.S. Pat. No. 9,349,183, Method and apparatus for three dimensionalviewing of images, U.S. Pat. No. 9,473,766, Method and apparatus forthree dimensional viewing of images, U.S. Pat. No. 9,980,691, Method andapparatus for three dimensional viewing of images, U.S. patentapplication Ser. No. 15/904,092, Processing 3D medical images to enhancevisualization, U.S. patent application Ser. No. 16/195,251, INTERACTIVEVOXEL MANIPULATION IN VOLUMETRIC MEDICAL IMAGING FOR VIRTUAL MOTION,DEFORMABLE TISSUE, AND VIRTUAL RADIOLOGICAL DISSECTION, U.S. patentapplication Ser. No. 16/524,275, USING GEO-REGISTERED TOOLS TOMANIPULATE THREE-DIMENSIONAL MEDICAL IMAGES, PCT/US19/47891, A VIRTUALTOOL KIT FOR RADIOLOGISTS, U.S. patent application Ser. No. 16/509,592,IMPLANTABLE MARKERS TO AID SURGICAL OPERATIONS, U.S. patent applicationSer. No. 16/563,985, METHOD AND APPARATUS FOR THE INTERACTION OF VIRTUALTOOLS AND GEO-REGISTERED TOOLS, and PCT/US19/23968, RADIOLOGIST-ASSISTEDMACHINE LEARNING WITH INTERACTIVE, VOLUME-SUBTENDING 3D CURSOR.

FIG. 2 illustrates an apparatus for implementing the process illustratedin FIG. 1. A radiologic imaging system 200 (i.e., digital breasttomosynthesis) is used to generate 2D medical images 202 of a breast204. The 2D medical images 202 are provided to an image processor 206,that includes processors 208 (e.g., CPUs and GPUs), volatile memory 210(e.g., RAM), and non-volatile storage 212 (e.g. HDDs and SSDs). Aprogram 214 running on the image processor implements one or more of thesteps described in FIG. 1. 3D medical images are generated from the 2Dmedical images and displayed on an IO device 216. The IO device mayinclude an extended reality display (e.g., mixed reality, virtualreality or augmented reality headset), monitor, tablet computer, PDA(personal digital assistant), mobile phone, or any of a wide variety ofdevices, either alone or in combination. The IO device may include atouchscreen, and may accept input from external devices (represented by218) such as a keyboard, mouse, joystick, and any of a wide variety ofequipment for receiving various inputs. However, some or all the inputscould be automated, e.g. by the program 214.

FIGS. 3A, 3B and 3C illustrate the breast with georegistration points onthe skin surface, georegistration grid on the skin surface andcoordinate system. FIG. 3A illustrates the breast 301 withgeoregistration points 302. FIG. 3B illustrates the breast 303 withgeoregistration points 304 and a georegistration grid 305. FIG. 3Cillustrates the breast 306 with georegistration points 307,georegistration grid 308 and the coordinate system 309.

FIGS. 4A and 4B illustrate the X-ray detector, x-ray beam, breast,microcalcification within the breast and the detector. In FIG. 4A, thex-ray tube is shown. The x-ray beam 402 is shown. The breast 403 isshown. 404 represents a microcalcification within the breast. 405represents the image acquired. 406 represents the microcalcificationwithin the image of the breast. 407 represents the image of the breast.In FIG. 4B, 408 represents the x-ray tube, which has been moved to a newposition. 409 represents the x-ray beam from this new position. 410represents the microcalcification in the breast, which has not moved.411 represents the breast, which has also not moved. 412 represents theimage acquired. 413 represents the image of the microcalcification,which has shifted in position as compared to 406 due to the new angle ofthe x-ray. 414 again demonstrates the image of the breast. 415represents the distance that the microcalcification has shifted in image412 as compared to image 405. Note that structures closer to thedetector have less shift within the image than structures farther fromthe detector.

FIGS. 5A and 5B illustrate the geometry required to compute the (x, y,z) coordinate of a microcalcification. In FIG. 5A, the geometry of theX-ray tube, X-ray detector, compression paddle, breast andmicrocalcification are illustrated. 501 illustrates the x-ray tube atthe initial position directly above the breast in a position to sendx-rays are emitted vertically toward the floor. 502 illustrates thex-ray tube at a subsequent position in a position to send x-raysobliquely toward the floor. 503 illustrates the path of X-ray photonspassing from the X-ray tube 502 through the microcalcification 507 andthen to the detector 508. Similarly, 504 illustrates the path of X-rayphotons passing from the X-ray tube 501 through the microcalcification507 to the detector 508. 505 illustrates the paddle that compresses thebreast, the properties of which have minimal X-ray attenuation. 506 isthe patient's breast containing the microcalcification 507. In FIG. 5B,the geometry of the microcalcification within the breast, the photonbeams from the two X-ray tube positions, the X-ray detectors isillustrated. 509 represents the location of the microcalcification,which will be assigned an (x, y, z) coordinate. The coordinate of themicrocalcification of interest 509 is located at the intersection of thevertically oriented X-ray beam 510 and the obliquely oriented X-ray beam513. The angle between the vertically oriented X-ray beam 510 and theobliquely oriented X-ray beam 513 is a 512. The spot where thevertically oriented X-ray beam hits the detector is 511. Note that theangle between the vertically oriented X-ray beam 510 and the detector isdenoted as 516 and is 90 degrees. The spot where the obliquely orientedX-ray beam (from 502 and 503 in FIG. 5A) hits the detector is 515. Thedistance between the spot where the vertically oriented X-ray beam hitsthe detector 511 and the spot where the obliquely oriented X-ray beamhits the detector is 506. The angle between the detector, a portion ofwhich is shown as 506, and the obliquely oriented X-ray beam 513 is β.To illustrate the calculation of the coordinates of themicrocalcification from this geometry, the following assumptions will bemade: 512 will be assumed to be 30 degrees; and, 506 will be assumed tobe 2 cm. According to the law of a right triangle, angle β will equal 60degrees. The tangent of 60 degrees equals the vertical height of themicrocalcification in the z-direction divided by the distance 506.Therefore, the height of the microcalcification over the detector is √12cm, which will be the z-coordinate of the microcalcification. Thex-coordinate of the microcalcification is the spot on the detector fromthe vertically oriented X-ray beam 510. The y-coordinate ofmicrocalcification is also the spot on the detector from the verticallyoriented X-ray beam 510. X, Y, Z axes 516 are shown. For example, if alinear calcification is oriented vertically, on the image wherein thex-ray beam is vertically oriented with respect to the floor, thecalcification would appear as a dot. If the x-ray beam is at an anglewith respect to the calcification, then the calcification would appearas a line.

FIGS. 6A through 6D illustrate a cluster of microcalcifications within avolume-subtending 3D cursor with external tissues subtracted,transparency of internal structures altered and tissues both inside andoutside of the 3D cursor subtracted except for the cluster ofmicrocalcifications. In FIG. 6A, 600 illustrates an artery. 601illustrates the breast tissue. 602 illustrates the 3D cursor. 603illustrates a cluster of microcalcifications. 604 illustrates a vein. InFIG. 6B, note that the tissues external to the 3D cursor 605 have beensubtracted. Note that the tissues inside of the 3D cursor 606 areunchanged from FIG. 6A. In FIG. 6C, the items shown are the same as FIG.6B with the exception that the transparency (or grayscale) of the arteryand vein 607 is altered to improve visualization of themicrocalcifications. Transparency can be performed in a variety of ways,such as sparse sampling of the voxels segmented to a particularstructure. Alternatively, or additionally, the opacity level of aparticular voxel can be altered. Alternatively, or additionally,prioritized volume rendering can be perform as described in U.S.Provisional Patent Application 62/846,770 A METHOD OF PRIORITIZED VOLUMERENDERING TO IMPROVE VISUALIZATION OF PRIORITIZED ITEMS WITHIN A 3DVOLUME. In FIG. 6D, all tissues both inside and outside of the 3D cursor608 with the exception of the microcalcifications are eliminated. Thisoverall effort aims to improve visualization of microcalcifications fora user wearing an extended reality headset. By rotating, zooming,tilting or turning one's head, converging, one can better visualize andunderstand the true 3D distribution of microcalcifications.

FIGS. 7A through 7D illustrate viewing of the isolated cluster ofmicrocalcifications using an augmented reality or virtual reality headmounted display. 700 illustrates the head display unit, which includesthe transmit/receive element 706, the inertial measurement unit 707 andthe georegistration point 708. 701 illustrates the left eye viewingperspective with left eye field of view. 702 illustrates the right eyeviewing perspective with right eye field of view. 703 illustrates the 3Dcursor. 704 illustrates the x, y, z coordinate system. 705 illustratesthe microcalcifications. FIG. 7A illustrates the initial viewingperspective. FIG. 7B illustrates changing the interocular distance andangular field of view (compare with FIG. 7A). FIG. 7C illustrates therotating the 3D cursor 90 degrees for a different viewing angle of thevolume of interest (compare with FIG. 7A and FIG. 7B). FIG. 7Dillustrates changing the viewing perspectives by rotating it by 90degrees, such as looking at the cluster of microcalcifications from theside rather than from the front.

FIGS. 8A through 8C illustrate the use of the geo-registered platformand geo-registered focal point pen to inspect the microcalcificationswithin the 3D cursor. 800 illustrates the geo-registered platform. 801illustrates a geo-registered point on the platform. 802 illustrates thetransmit/receive element. 803 illustrates the inertial measurement unit.804 illustrates the 3D cursor. 805 illustrates the microcalcificationswithin the 3D cursor. Note that the microcalcifications are shown ascubes for illustrative purposes only. In actuality, themicrocalcifications may take on various shapes, sizes and densities. InFIG. 8A, the hand-held georegistration platform is affixed to the 3Dcursor, which enables the mammographer to translate and rotate thegeoregistration platform and therefore view the 3D cursor and cluster ofmicrocalcifications within the 3D cursor from multiple angles. In FIG.8B, the hand-held georegistration platform is tilted and the 3D cursorand cluster of microcalcifications within the 3D cursor are likewisetilted. In FIG. 8C, the georegistered focal point pen 806 including thecomponents of the transmit/receive element 807 and the inertialmeasurement unit 808 and the georegistered point 809 is shown pointingto the microcalcifications 805 within the 3D cursor 804. Thegeoregistered pen can be used to perform functions such as image markupfor communication with other physicians or areas of concern.

FIG. 9 illustrates a top down view of the mammographer's deskillustrating several of the tools with position and orientationtracking. 900 illustrates the standard equipment at the radiologistswork station including multiple monitors, computer, power supply, mousekeyboard, voice dictation, etc. 901 illustrates the head display unit(HDU), such as an augmented reality, mixed reality or virtual realitysystem equipped with a transmit/receive element, an inertial measurementunit and a georegistration point. 902 illustrates the real-world imageof the georegistered platform that the user can visualize with the HDU.903 illustrates the virtual image of the 3D cursor displayed on the HDU,which includes the 3D cursor and a cluster of microcalcifications. 904illustrates the master control platform used to guide thegeoregistration system. 905 illustrates the mammographer's left handholding the georegistration platform, which is equipped with atransmit/receive element, an inertial measurement unit and ageoregistration point. 906 illustrates the mammographer's right handholding a focal point pen, which is equipped with a transmit/receiveelement, an inertial measurement unit and a georegistration point.

FIG. 10 illustrates the conversion of a digital breast tomosynthesisdataset into a single voxelated dataset. 1000 illustrates a firstdigital breast tomosynthesis image from a first position of the x-raytube. 1002 illustrates a first digital breast tomosynthesis image from afirst position of the x-ray tube. 1004 illustrates a first digitalbreast tomosynthesis image from a first position of the x-ray tube. 1006illustrates a first digital breast tomosynthesis image from a firstposition of the x-ray tube. 1008 illustrates a voxelated dataset. Aseries of pixels corresponding to calcifications would be detected on afirst image 1000. From those pixels, a back projected image would beperformed to the x-ray tube position. For each pixel with acalcification density, a set of potential x,y,z coordinates for the truelocation of the calcifications is yielded (i.e., along a line from thedetector location to the x-ray tube). Next, a set of pixelscorresponding to calcifications would be detected on a second image1002. From those pixels, a back projected image would be performed tothe x-ray tube position and a second set of potential x,y,z coordinatesfor the true location of the calcifications is yielded (i.e., along aline from the x-ray detector to the x-ray tube). The x,y,z coordinate ofthe calcification would be determined by the intersection point from thefirst tomosynthesis image to the second tomosynthesis image. Thisprocess would be repeated and a voxelated dataset formed for each pairof images. Note that multiple voxelated datasets could therefore beformed. Thresholds to plot only the calcifications could be set.Alternatively, thresholds to plot all tissues could be set.

FIGS. 11A through 11D illustrate variable compression during digitalbreast tomosynthesis. The breast 1100 is shown. The detector 1102 isshown. The compression paddle 1104 is shown. FIG. 11A illustrates a lowpressure of compression of the breast 1100. Note the initial contour ofthe breast 1100. FIG. 11B illustrates a high level of compression of thebreast 1100. Note the subsequent contour of the breast 1100. FIG. 11Cillustrates an initial configuration of a breast mass 1106. FIG. 11Dillustrates a subsequent configuration of a breast mass 1108. Note thatdifferent tissues in the body will have different tissue typeproperties. For example, bone is rigid. Fat is easily deformable.Cancers can be hard. The purpose of doing the compression at twodifferent levels is therefore to see how a lesion changes in itsappearance and configuration under the two different conditions. This isaccomplished by performing two different DBT examinations underdifferent levels of compression, generating a 3D volume at the firstlevel of compression, generating a 3D volume at the second level ofcompression and viewing the two datasets to see how the internalstructures would change. For example, there might be two adjacentlesions and the radiologist is unsure whether these lesions areconnected as a single bilobed mass or whether they are not connected atall. Such a technique can distinguish between these two scenarios. Anoptimum viewing strategy is to perform segmentation and then watch asegmented structure of interest change over the two (or more)configurations. Alternatively, compression could be performed fromside-to-side under two (or more) compression levels. Sometimes there canbe many lesions in a breast and it can be difficult for the radiologistto determine which lesion on the top-compression view corresponds towhich lesion on the side-compression view. This is where the skinmarkers may be of benefit. By utilizing skin markers, the internalstructures can be better analyzed. For example, there could be fourareas in the breast of interest to the radiologist. The radiologistcould place a 3D cursor over the first area of concern and then view itunder various external compression settings to determine how it changes.Then perform a similar process to the second lesion and so on. Analternative approach to DBT for breast imaging is breast MM. In thiscase, a dynamic compression device can be built and synchronized withthe breast MRI acquisition such that multiple volumes are obtained. Forexample, during a first time period at a first compression level, afirst volume is built. During a second time period at a secondcompression level, a second volume is built. And, so on. The volumes canbe viewed in a dynamic fashion on an extended reality display. Ideally,at least in some implementations, this would be performed with a fastacquisition MM. For example, a round mass might stay round even underdramatically different compression levels and this may indicate thehardness of the tumor and serve as an indicator for a first type oftumor (e.g., cancer risk). Alternatively, a round mass might becomeflattened into a pancake-like shape under a high amount of compressionand this may indicate that the tumor is soft and serve as an indicatorfor a different type of tumor. Dynamic compression devices can besubsequently designed and built to accommodate the process outlined inthis patent. Still further, the change in configuration of an anatomicfeature can be utilized to help assign a tissue-type property (e.g.,hardness). For example, a sphere shaped structure that remains sphereshaped despite a high pressure exerted upon it would be assigned a hardtissue-type property. In contrast, a sphere shaped structure thatbecomes pancake shaped when the same high pressure is exerted upon itwould be assigned a soft tissue-type property. This is further describedin U.S. patent application Ser. No. 15/904,092, INTERACTIVE VOXELMANIPULATION STRATEGIES IN VOLUMETRIC MEDICAL IMAGING ENABLES VIRTUALMOTION, DEFORMABLE TISSUE, AND VIRTUAL RADIOLOGICAL DISSECTION.Alternatively, this process could be performed in applications otherthan the breast. For example, a 3D imaging examination of the knee jointcould be performed at a first configuration wherein the knee is in astraight 180 degree position. Then, a 3D imaging examination of the kneejoint could be performed at a second configuration wherein the knee isslightly flexed to a 135 degrees position. Then, a radiologist wearingan extended reality head display unit can view the anatomic feature ofthe knee joint under different configurations and analyze changestherein. For example, a small meniscal tear can be obscured in the first180 degree extended position, but then identified when the knee is in a135 degree flexed position. MM coils can be manufactured to accommodatethese changes in positions and optimize imaging parameters. In additionto the analysis of motion of joints, this process could be performed toanalyze other structures that move in the body. The higher number ofvolumes would allow an improved ability to assess changes inconfiguration and would yield a more accurate analysis. For example,this process can be applied to an vascular structure whose configurationchanges over the cycles of systole and diastole. For example, thisprocess could be performed on the brain whose configuration changes overthe cycles of CSF pulsation. For example, this process could beperformed on the trachea or airways whose configuration changes over thecycles of inhalation and exhalation. The human body is mobile; thus,virtually every structure in the human body would be better analyzedduring the 3D analysis of motion, changes in configuration anddeformation as described in processes outlined in this patent.

FIGS. 12A through 12D illustrate two cases of tumors of varying hardnesslevels. FIG. 12A illustrates case 1 with a low level of compression. Thebreast 100, breast mass 1201, detector 1202, compression device 1203 andan initial pressure 1204 (e.g., with units of PSI) is illustrated. Notethe configuration of the breast 1200 is somewhat round and theconfiguration of the mass 1201 is round. FIG. 12B illustrates case 1with a high level of compression. The breast 1200, breast mass 1201,detector 1202, compression device 1203 and an subsequent higher pressure1205 (e.g., with units of PSI) is illustrated. Note the configuration ofthe breast 1200 is now more flattened (compare with FIG. 12A) and theconfiguration of the mass 1201 is still round. This would indicate thatthe mass 1201 is hard. A deformability index could be established toindicate the amount of deformation of a given tissue in relation to theamount of pressure. FIG. 12C illustrates case 2 with a low level ofcompression. The breast 1206, breast mass 1207, detector 1208,compression device 1209 and an initial pressure 1210 (e.g., with unitsof PSI) is illustrated. Note the configuration of the breast 1206 issomewhat round and the configuration of the mass 1207 is round. FIG. 12Dillustrates case 2 with a high level of compression. The breast 1206,breast mass 1207, detector 1208, compression device 1209 and ansubsequent higher pressure 1211 (e.g., with units of PSI) isillustrated. Note the configuration of the breast 1206 is now moreflattened (compare with FIG. 12C) and the configuration of the mass 1207is still flattened. This would indicate that the mass 1201 is soft anddeformable. A deformability index could be established to indicate theamount of deformation of a given tissue in relation to the amount ofpressure. Note the deformability index of the breast mass 1201 in case 1would be different from the deformability index of the breast mass 1207in case 2. Note that in this illustration, two compression levels areillustrated. A 3D breast imaging examination would be performed at eachof the two compression levels. Note that additional 3D imagingexaminations could be performed at varying compression levels (e.g., 0.5psi, 1.0 psi, 1.5 psi, 2.0 psi, 2.5 psi, 3.0 psi, 3.5 psi, 4.0 psi,etc.). The volumes will be reconstructed and viewed on an extendedreality head display unit.

FIG. 13 illustrates a digital breast tomosynthesis dataset performedwith skin markers. The breast 1300, breast mass 1301, skin radioopaquemarkers 1302, detector 1304, compression device 1305 with adjustablecompression 1306 are shown. The 3D cursor 1303 can be viewed on theimage processing workstation. The remainder of the elements of a digitalbreast tomosynthesis machine is not shown. Note that the skin markerscan be utilized as landmarks and reference points to improveunderstanding of how internal structures change. Note that this isespecially important with less 3D imaging volumes obtained.

What is claimed:
 1. A display unit comprising: a processor; an inertialmeasurement unit configured to determine an orientation of the displayunit; a transmit/receive element configured to receive signals from aset of transmitters in an area wherein the received signals are used tocompute a location of the display unit within the area; a left eyedisplay operably connected to the processor; a right eye displayoperably connected to the processor; and a non-transitory memoryconfigurable to have computer-executable instructions stored thereupon,which when executed by the processor cause the display unit to display a3D volume stereoscopically to a user wherein a left eye image isdisplayed on the left eye display and a right eye image is displayed onthe right eye display wherein the location of the display unit withinthe area changes to a subsequent location closer to the 3D volume;wherein the transmit/receive element of the display unit receivessignals from the set of transmitters in an area; wherein the processorcomputes the subsequent location of the display unit within the area;wherein a zoomed in left eye image is determined by the subsequentlocation of the display unit, the orientation of the display unit, aleft eye viewpoint, a left eye viewing angle, a location of the 3Dvolume within the area and an orientation of the 3D volume; wherein azoomed in right eye image is determined by the subsequent location ofthe display unit, the orientation of the display unit, a right eyeviewpoint, a right eye viewing angle, the location of the 3D volumewithin the area and the orientation of the 3D volume; wherein the zoomedin left eye image is displayed on the left eye display; and wherein thezoomed in right eye image is displayed on the right eye display.
 2. Thedisplay unit of claim 1 further comprising: wherein the left eye imageis determined by the location of the display unit, the orientation ofthe display unit, a left eye viewpoint, a left eye viewing angle, alocation of the 3D volume and an orientation of the 3D volume; andwherein the right eye image is determined by the location of the displayunit, the orientation of the display unit, a right eye viewpoint, aright eye viewing angle, the location of the 3D volume and theorientation of the 3D volume.
 3. The display unit of claim 1 furthercomprising geo-registering the 3D volume to a first tangible object inthe area comprising: wherein the first tangible object has atransmit/receive element configured to receive signals from the set oftransmitters in an area wherein a location of the first tangible objectwithin the area is computed; wherein the 3D volume is affixed to thefirst tangible object; wherein the 3D volume has location coordinates inthe area; wherein a change in the first tangible object's positioncauses a corresponding change in the 3D volume's position; and wherein achange in the first tangible object's orientation causes a correspondingchange in the 3D volume's orientation.
 4. The display unit of claim 1further comprising: wherein a location of the 3D volume changes withinthe area changes to a subsequent location closer to the display unit;wherein a zoomed in left eye image is determined by the location of thedisplay unit, a left eye viewpoint, a left eye viewing angle, thesubsequent location of the 3D volume within the area and an orientationof the 3D volume; wherein a zoomed in right eye image is determined bythe location of the display unit, a right eye viewpoint, a right eyeviewing angle, the subsequent location of the 3D volume within the areaand the orientation of the 3D volume; wherein the zoomed in left eyeimage is displayed on the left eye display; and wherein the zoomed inright eye image is displayed on the right eye display.
 5. The displayunit of claim 1 further comprising: wherein a first tangible object hasa transmit/receive element configured to receive signals from the set oftransmitters in an area; wherein the processor computes a location ofthe first tangible object within the area; and wherein the movement ofthe first tangible object on a portion of the 3D volume causes anannotation on the 3D volume.
 6. The display unit of claim 1 furthercomprising a geo-registration point.
 7. The display unit of claim 1further comprising a laser range finder.
 8. A display unit comprising: aprocessor; an inertial measurement unit configured to determine anorientation of the display unit; a transmit/receive element configuredto receive signals from a set of transmitters in an area wherein thereceived signals are used to compute a location of the display unitwithin the area; a left eye display operably connected to the processor;a right eye display operably connected to the processor; and anon-transitory memory configurable to have computer-executableinstructions stored thereupon, which when executed by the processorcause the display unit to display a 3D volume stereoscopically to a userwherein a left eye image is displayed on the left eye display and aright eye image is displayed on the right eye display wherein a locationof the 3D volume changes within the area changes to a subsequentlocation closer to the display unit; wherein a zoomed in left eye imageis determined by the location of the display unit, a left eye viewpoint,a left eye viewing angle, the subsequent location of the 3D volumewithin the area and an orientation of the 3D volume; wherein a zoomed inright eye image is determined by the location of the display unit, aright eye viewpoint, a right eye viewing angle, the subsequent locationof the 3D volume within the area and the orientation of the 3D volume;wherein the zoomed in left eye image is displayed on the left eyedisplay; and wherein the zoomed in right eye image is displayed on theright eye display.
 9. The display unit of claim 8 further comprising:wherein the left eye image is determined by the location of the displayunit, the orientation of the display unit, a left eye viewpoint, a lefteye viewing angle, a location of the 3D volume and an orientation of the3D volume; and wherein the right eye image is determined by the locationof the display unit, the orientation of the display unit, a right eyeviewpoint, a right eye viewing angle, the location of the 3D volume andthe orientation of the 3D volume.
 10. The display unit of claim 8further comprising geo-registering the 3D volume to a first tangibleobject in the area comprising: wherein the first tangible object has atransmit/receive element configured to receive signals from the set oftransmitters in an area wherein a location of the first tangible objectwithin the area is computed; wherein the 3D volume is affixed to thefirst tangible object; wherein the 3D volume has location coordinates inthe area; wherein a change in the first tangible object's positioncauses a corresponding change in the 3D volume's position; and wherein achange in the first tangible object's orientation causes a correspondingchange in the 3D volume's orientation.
 11. The display unit of claim 8further comprising: wherein the location of the display unit within thearea changes to a subsequent location closer to the 3D volume; whereinthe transmit/receive element of the display unit receives signals fromthe set of transmitters in an area; wherein the processor computes thesubsequent location of the display unit within the area; wherein azoomed in left eye image is determined by the subsequent location of thedisplay unit, the orientation of the display unit, a left eye viewpoint,a left eye viewing angle, a location of the 3D volume within the areaand an orientation of the 3D volume; wherein a zoomed in right eye imageis determined by the subsequent location of the display unit, theorientation of the display unit, a right eye viewpoint, a right eyeviewing angle, the location of the 3D volume within the area and theorientation of the 3D volume; wherein the zoomed in left eye image isdisplayed on the left eye display; and wherein the zoomed in right eyeimage is displayed on the right eye display.
 12. The display unit ofclaim 8 further comprising: wherein a first tangible object has atransmit/receive element configured to receive signals from the set oftransmitters in an area; wherein the processor computes a location ofthe first tangible object within the area; and wherein the movement ofthe first tangible object on a portion of the 3D volume causes anannotation on the 3D volume.
 13. The display unit of claim 8 furthercomprising a geo-registration point.
 14. The display unit of claim 8further comprising a laser range finder.